Crate datafusion_row
source · [−]Expand description
This module contains code to translate arrays back and forth to a
row based format. The row based format is backed by raw bytes
([[u8]
]) and used to optimize certain operations.
In general, DataFusion is a so called “vectorized” execution
model, specifically it uses the optimized calculation kernels in
arrow
to amortize dispatch overhead.
However, as mentioned in this paper, there are some “row oriented” operations in a database that are not typically amenable to vectorization. The “classics” are: hash table updates in joins and hash aggregates, as well as comparing tuples in sort / merging.
Re-exports
pub use layout::row_supported;
pub use layout::RowType;
Modules
RowAccessor
provides a Read/Write/Modify access for row with all fixed-sized fields:
Various row layouts for different use case
read_as_batch
converts raw bytes to RecordBatch
RowWriter
writes RecordBatch
es to Vec
Macros
Structs
Columnar Batch buffer that assists creating RecordBatches